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Abstract. By a recent result obtained by R. Howlett and the author 
considerable progress has been made towards a complete solution of 
the isomorphism problem for Coxeter groups. In this paper we give a 
survey on the isomorphism problem and explain in particular how the 
result mentioned above reduces it to its 'reflection preserving' version. 
Furthermore we desreibe recent developments concerning the solution of 
the latter. 



1 Introduction 

Coxeter groups are important in several mathematical areas. It is therefore a 
bit surprising that the isomorphism problem for those groups does not seem to have 
been considered before the late 1990's. They only earlier reference known to the 
author where this problem has been asked is |17j . The first major contributions to 
it are 1161 and jlOj . In j!6) a rigidity result is proved for a certain class of Coxeter 
groups. Rigidity means that the Coxeter generating sets are all conjugate. In II 0| 
diagram twists have been introduced. Those provide non-trivial examples of non- 
rigid Coxeter groups. The question about which Coxeter systems are rigid arises 
naturally as well as the more general question about the isomorphism problem for 
Coxeter groups. 

The purpose of the present paper is to give a survey about what is known at 
present about the isomorphism problem. The main motivation for writing this sur- 
vey is provided by a recent result obtained by the author in collaboration with Bob 
Howlett. This result reduces the isomorphism problem to its 'reflection-preserving 
version'. For the solution of the latter there is a conjecture stated in |10| . Consid- 
erable progress towards a proof of this conjecture was made in [55] and by recent 
work of Pierre-Emmanuel Caprace in |13| there is reasonable hope that this con- 
jecture will be proved in the near future. Due to these facts there is now a clear 
picture of what the solution of the isomorphism problem should look like. In fact, 
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at present there is a solution if one assumes that there are no irreducible spherical 
residues of rank 3. We state two conjectures in Sectional The first one is known 
to be true for all Coxeter systems having no H3-subsystems; the second is a re- 
finement of Conjecture 8.1 in jlOj already mentioned. Under the assumption that 
both conjectures are true, we give an algorithm for the solution of the isomorphism 
problem. 

Two versions of the isomorphism problem. Let W be a group and let 
S C W be a set of involutions. Then M(S) denotes the square matrix (o(ss')) s>s t e g 
where o(w) denotes the order of an element w € W. The matrix M(S) is called 
the type of S. As the elements of S are involutions we have the following. 

1. For all s,s' <E S we have o(ss') <G N U {oo}; 

2. for all s ^ s' G S we have o(ss') — o(s's) > 2; 

3. for all s E S we have o(ss) = 1. 

Hence, the matrix M(S) is a symmetric square matrix with entries in the set 
N U {oo} where all entries on the main diagonal are equal to one and all remaining 
entries are strictly greater than one. Such a matrix is called a Coxeter matrix over 
S. 

Let (W, S) be as above. We call (W, S) a Coxeter system (of type M(S)) if 
(S) = W and if the relations ((ss')°' ss ) = lw)*,»'eS provide a presentation of W. 
For a given Coxeter matrix M = (My),^/ over a set /, we define the Coxeter 
group of type M by setting W{M) := (I | {[ij) miJ = l)ij e i). It is a basic fact that 
the pair (W(M),I) is a Coxeter system of type M (i.e. that o(ij) = rriij in W(M) 
for all i, j € I). 

In this paper we will consider the isomorphism problem for finitely generated 
Coxeter groups. Thus, if we talk about a Coxeter system (W, S) or a Coxeter matrix 
M over / it is always understood that the sets S and I are finite. 

Here are two versions of the isomorphism problem for Coxeter groups. 

Problem 1 Given two Coxeter matrices M and M', decide whether the groups 
W(M) and W(M') are isomorphic. 

Problem 2 Given two Coxeter matrices M and M', hnd all isomorphisms from 
W(M) onto W(M'). 

At first sight, Problem 1 seems to be a more natural question than Problem 
2. The latter is just a more general version of the first. Roughly speaking, the 
solution of Problem 2 is equivalent to the solution of Problem 1 and a description 
of the automorphism group of W(M) for any Coxeter matrix M. This is in fact 
the main motivation to consider Problem 2. It turns out that for certain Coxeter 
matrices M a good understanding of the automorphism group of the group W{M) 
is only possible if a solution of Problem 2 is available for all Coxeter matrices M' . 

Content. In Section [3 we recall some definitions, fix notation and mention 
some basic facts concerning Coxeter groups. In Section we will consider the 
rigidity problem for Coxeter groups. This is an interesting special case of the 
isomorphism problem. In this section we will provide examples of non-rigid Coxeter 
systems which will play an important role later. Section 0] is devoted to explaining 
the results obtained in |23j and |27| and how these results reduce the isomorphism 
problem to its 'reflection-preserving version' which will then be treated in Section 
In Section we explain an algorithm to solve the isomorphism problem under 
the assumption that Conjectures 1 and 2 of Section [S] hold. Finally, in Section 
we will make some remarks on the automorphism groups of Coxeter groups. 
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Remark: It was mentioned above that there is no contribution to the isomorphism 
problem for Coxeter groups before the late 1990's. Since then, however, there are 
several publications concerning this subject. For instance, Problem 1 has been 
solved completely in the case where M is assumed to be even (i.e. no odd entries) 
by P. Bahls and M. Mihalik (see |34| and the references given there). 

In this survey paper we do not attempt to give a systematic description of 
all contributions to the isomorphism problem for Coxeter groups. We mention 
results (or consequences of them) whenever it will be convenient. However, we 
try to include all references on the subject in the bibliography. Thus, quite a few 
references will be mentioned only there. 

Acknowledgement. The content of this paper is based on my talk at the 
Coxeter Legacy Conference at Toronto in May 2004. I thank the organizers for the 
invitation to present this survey at this conference. 

2 Preliminaries 

Coxeter diagrams. With a Coxeter matrix M = (mij)ij £ j we associate 
its diagram T(M). It is the edge-labelled graph (I,E(M)), where the edge-set 
is E(M) :— {{i, j} \ rriij > 3} and where an edge {i, j} € E(M) has the label rriij. 
We do not distinguish between a Coxeter matrix and its diagram since they carry 
the same information. We call a Coxeter matrix irreducible if its associated Coxeter 
diagram is connected. An irreducible component of M is a subset J of /, which is 
a connected component of the diagram. A Coxeter matrix M is called spherical 
if W(M) is finite. The irreducible spherical Coxeter diagrams have been classified 
by H.S.M. Coxeter in |18j : we will use the Bourbaki notation for denoting them 
with the exception that we denote rank 2 diagrams for the dihedral groups of order 
2n by l2(n). Thus we have the four series A n ,C n = B n ,D n and I^in) and the 6 
exceptional diagrams Eq, £7, Eg, F4, H3 and H4. 

An isomorphism from a Coxeter diagram M — {mij)i,j£i onto a Coxeter dia- 
gram M' = (m'ij)i,jEi' is a graph isomorphism which preserves the edge-labels. 

Let M = (mij)i^ci be a Coxeter matrix over I and let J be a subset of /. 
Then we put Mj := (mjk)j,k£j and := {k G I \ rrikj — 2 for all j G J}. 

A Coxeter matrix M is called right-angled if all edge-labels of T(M) are infinite; 
it is called 2- spherical if there are no infinities; it is called even if there are no odd 
labels and it is called of large type if the diagram is a complete graph (hence if there 
are no 2's in M). 

Coxeter systems. Let (W, S) be a Coxeter system. The set of its reflections 
is defined to be the set S w := {wsw" 1 \ s € S and w £ W}. The length of w G W 
is the length of a shortest product of elements in S representing w; it is denoted 
by l(w). We call (W, S) right-angled, 2-spherical, even or of large type if this is the 
case for M(S). 

We list some facts about Coxeter systems which are important in the sequel. 
Facts 1 and 2 are basic and can be found in any standard reference on Coxeter 
groups (see jH] or 29 ); Fact 3 is a non-trivial exercise in [2] but it follows also 
from the fact that the Davis-complex of a Coxeter system is CAT(O); Fact 4 is 
contained in |44| ; Fact 5 can be shown by considering the geometric representation 
and Fact 6 is just an easy consequence of the definition of a Coxeter system. 

1. If J C S, then ((J), J) is a Coxeter system. 



4 



Bernhard Miihlherr 



2. Let J C S and I : W -> N be the length function of (W,£). Then the 
following are equivalent: 

a) ((J), J) is finite; 

b) there is an element pj such that > l(x) for all ^ / i£ (J). 
Moreover, if these two conditions are satisfied, then pj = lyy- 

3. If X < W is a finite subgroup, then there exist w £ W and J C S such that 
X" 1 < (J) and such that J is a spherical subset of S (i.e. (J) finite). 

4. Let r S W be an involution. Then there exist w £ W and J C S such that 
J is spherical, wpjw^ 1 = r and such that pj is central in (J). 

5. Suppose that J is a spherical subset of S such that pj is central in (J). 
Then the normalizer of (J) in W and the centralizer of p,j in Ly coincide. 

6. Let (W, 5 1 ) be a Coxeter system. Then each permutation 7r of 5 which is an 
automorphism of M (S) extends uniquely to an automorphism 7^ of W. 

Let (W, S) be a Coxeter system. By Fact 6 we can identify the stabilizer of S in 
Aut(W / ) with the group of automorphisms of M(S); this subgroup will be denoted 
by Ts{W) and its elements are called the graph- automorphisms of (W, S). The 
group Ts has trivial intersection with the group Inn(VF) of inner automorphisms. 
An automorphism of W will be called inner-by- graph if it can be written as a 
product of an inner automorphism and a graph-automorphism. 

3 Rigidity 

Let G be a group and R C G a set of involutions. Recall that the Coxeter 
matrix M(R) is called the type of R; the set R is called universal if ((R),R) is 
a Coxeter system; it is called a Coxeter generating set of G if it is universal and 
G=(R). 

A Coxeter matrix M is called rigid if for each Coxeter generating set R of 
W(M) the Coxeter diagrams M(R) and M are isomorphic. It is called strongly 
rigid if any two Coxeter generating sets of W(M) are conjugate in W(M). 

Clearly, strong rigidity implies rigidity. If a Coxeter diagram is (strongly) rigid, 
then we call the corresponding Coxeter group and Coxeter system (strongly) rigid 
as well. 

If one can show that the Coxeter diagram M of Problem 1 is rigid, then this 
problem is trivially solved. The answer is just that the Coxeter diagram M' has to 
be isomorphic to M. 

Similarly, if one can show that the Coxeter diagram M is strongly rigid, then 
Problem 2 is solved. An isomorphism onto W(M') exists if and only if M' and M 
are isomorphic. Moreover, the automorphism group of W(M) is just the semi- 
direct product of the group of inner automorphisms with the group of graph- 
automorphisms of W(M); in other words: all automorphisms of W are inner-by- 
graph. 

There are several interesting classes of Coxeter systems which are not rigid. 
Before describing them we present some positive results. The first is due to D. 
Radcliffe |43] , 

Theorem 3.1 Right-angled Coxeter systems are rigid. 

Although we fixed the convention that all Coxeter systems in this paper are 
by definition of finite rank it is appropriate to mention that the theorem above has 
been generalized to right-angled Coxeter systems of arbitrary rank by A. Castella 
(see |15p. The next result about strong rigidity is the result of R. Charney and M. 
Davis already mentioned in the introduction (see |16|L 
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Theorem 3.2 Let (W, S) be a Coxeter system. If W is capable of acting 
effectively, properly and cocompactly on some contractible manifold, then (W, S) is 
strongly rigid. In particular, Coxeter groups of affine and compact hyperbolic type 
are strongly rigid. 

The next result is very recent. An important step towards a proof of it was 
already made in I28| ; in the version presented here it is a consequence of the main 
results in |14| and |23| . 

Theorem 3.3 Suppose that (W, S) is irreducible, non-spherical and 2-spherical, 
then (W, S) is strongly rigid. 

In the following we describe two ways to manipulate the generating set of a 
given Coxeter system in order to produce a new one whose type is possibly non- 
isomorphic to the type of the original one. It is conjectured (and known to be true 
in a lot of special cases) that Coxeter systems are rigid up to these manipulations. 

Pseudo-Transpositions. Let k > 1 be a natural number and put n := 2(2fe+l).| 

We consider the dihedral group W of order 2n as the group of isometries preserving 
a regular n-gon in the euclidian plane. Let s,t £ W be two reflections whose axes 
intersect in an angle ^ and let p be the central symmetry. Then it is easily seen 
that {s, t} and {s, tst, p) are both Coxeter generating sets for W of type hin) and 
Ii(2k + 1) x A\ respectively. Thus, the dihedral group of order 2n is a non-rigid 
Coxeter group because it has two Coxeter generating sets of different types. This 
example is of course trivial and a bit cheating because one of the two Coxeter ma- 
trices is not irreducible. However, it can be used to produce more general examples 
by taking direct products or free products. In 27^ pseudo-transpositions have been 
introduced in order to describe the general feature. 

Let (W, S) be a Coxeter system and let r G S. We call t a pseudo-transposition 
if the following holds. 
PT1 There is a unique t £ S such that o(rt) = 2(2k + 1) for some natural number 
k > 1. 

PT2 For all s e S\{r,t} one has o(rs) G {2, oo} and if o(sr) = 2, then o(si) = 2 
as well. 

The following is an easy observation about pseudo-transpositions. 

Lemma 3.4 Let (W, S) be a Coxeter system, letr G S be a pseudo-transposition 
of (W, S) and let t G S be as in the definition above. Then S \ {r} U {rtr, p^ T t j} is 
a Coxeter generating set ofW. 

There is also another kind of pseudo-transpositions for Coxeter systems based 
on the fact that the Coxeter groups W(C n ) and W(D n x A\) are isomorphic for 
odd n. They yield also non-isomorphic Coxeter generating sets in a similar way. 
We refer to [22] for the details. 

Let (W, S) be a Coxeter system, let r G S be a pseudo-transposition and let R 
be the 'new' Coxeter generating set as described in the lemma above. Then we call 
the Coxeter system (W,R) an elementary reduction of (W, S). A Coxeter system 
(W, S') will be called a reduction of (W, S) if it can be obtained from (W, S) by 
a sequence of elementary reductions. Finally, we call (W, S) reduced, if there are 
no pseudo-transpositions. It is easy to see that each Coxeter system has a reduced 
reduction. 

Given a Coxeter diagram M over a set 7, then a Coxeter diagram M' over I' 
is called an elementary reduction of M if there is an elementary reduction of the 
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Coxeter system (W(M), I) whose type is isomorphic to M'; we call M' a reduction 
of M if M' can be obtained from M by a sequence of elementary reductions and 
we call M reduced if the system (W(M), I) has no pseudo-transpositions. 

Clearly, any rigid Coxeter system has to be reduced in view of Lemma 13.41 
above. The following result is due to M. Mihalik |34) and is based on earlier work 
of P. Bahls pQ; it states that the converse is true for even Coxeter systems. 

Theorem 3.5 An even Coxeter system is rigid if and only if there is no pseudo- 
transposition. 

Note that this result generalizes Theorem 13. II 

Twistings. In this subsection we describe twistings as they were introduced 
in |10| and we give some further definitions concerning them. 

Let (W, S) be a Coxeter system and let J,KES. We call the pair (J, K) an 
S -admissible pair if the following holds. 

AD1 J is a spherical subset of S and K n ( J U J- 1 ) = 0. 

AD2 For all k E K and / E L := S \ ( J U J x U K) the order of kl is infinite. 

An S-admissible pair (J, K) is called trivial if K or L are empty. For a in- 
admissible pair (J, K) we put T {J _ K) {S) := J U J x U K U {pj^j \l EL}. 
The following lemma is not too difficult to prove (see |10|). 

Lemma 3.6 Let (W, S) be a Coxeter system and let (J, K) be a S -admissible 
pair. Then T(jx)(S) is a Coxeter generating set of W which is contained in S w . 

Let (W,S), (J,K) and S' :— Trj^x){S) be as in the previous lemma. If pj is 
central in (J), then it is easily verified that M(S) is isomorphic to M(S'). If p,j is 
not central in (J), then M(S) is not isomorphic to M(S') in the generic case. The 
following example of such a situation was given in |37| . 

Example: Let (W, S) be a Coxeter system such that S — {si, S2, S3, S4} and such 
that o(siS2) = o(s2Sz) = o(saS4,) = 3 and o(siS3) = o(siSi) — o(s2S4) = 00. We 
put J := {s2, S3} and K := {si}. It follows that 

S' := T( Jt K)(S) ■= {s'j := si,s' 2 := s 2 , s 3 := S3, s' 4 := S2S3S2S4S2S3S2} 

and that o(sis' 2 ) — o(s' 2 s' 3 ) = o(s 2 S4) = 3 and o(s'±s' 3 ) = ois^s^) — o(sgS4) = 00. 
Thus M(S) and M(S') are not isomorphic. 

Let S, R be Coxeter generating sets of a group W; we call R a twist of S if there 
is a S'-admissible pair (J, K) such that R — T^ JK ^{S). It is readily verified that R 
is a twist of 5 if and only if S is a twist of i? and that S w = R w in this case. A 
Coxeter generating set S is called twist-rigid if there are no non-trivial S'-admissible 
pairs; i.e. if there are no twists of S which are not conjugate to S in W. 

Let M be a Coxeter matrix over /. A Coxeter matrix M' is called a twist of 
M if there is a twist /' of / in the Coxeter system (W(M),I) such that M(J') is 
isomorphic with M'. As before one verifies that M' is a twist of M if and only if 
M is a twist of M'. 

We close this section with a result about strong rigidity for Coxeter groups. 
Obviously, if (W, S) is a strongly rigid Coxeter system, then S has to be twist-rigid. 
The following theorem provides the converse under the additional assumption that 
all Coxeter generating sets R of W are contained in S w . In view of Corollary 14. 21 
below there are 'a lot of examples' where this assumption holds. 
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Theorem 3.7 Let M be a non- spherical, irreducible Coxeter diagram over I 
such that there is no subdiagram of type H3 . Suppose that I is a twist-rigid subset 
ofW(M) and that all Coxeter generating sets ofW(M) are contained in J W ( M ) , 
Then M is strongly rigid. 

This theorem was first proved in the large-type case (mjj > 2 for all i,j) in 
|38| : the result as it is stated above has been obtained recently by P.-E. Caprace 

m 

4 The reduction to the restricted isomorphism problem 

The restricted isomorphism problems for Coxeter groups are the following: 
Problem 3: Given a Coxeter system (W, S) and a Coxeter matrix M , decide 
whether there is a Coxeter generating set R C S w ofW such that M(R) = M . 

Problem 4: Given a Coxeter system (IT, S) and a Coxeter matrix M , End all 
Coxeter generating sets R C S w ofW with M(R) = M. 

In |27| Problems 1 and 2 of the introduction have been reduced to Problems 3 
and 4 respectively. This reduction is based on the results on the finite continuation 
of a reflection in a Coxeter group, which have been obtained in [23) . The purpose 
of this section is to describe the results obtained in both references. The original 
motivation for the investigations in |23| was to find a tool to characterize reflections 
in abstract Coxeter groups. We first provide some examples, where an abstract 
Coxeter group does not determine 'its set of reflections'. 

We have already seen examples, where an abstract Coxeter group has different 
Coxeter generating sets yielding different sets of reflections. If (W, S) is not reduced 
and if R is an elementary reduction of S, then S w % R w and R w % S w '. We will 
now obtain further examples by producing automorphisms of Coxeter groups which 
do not preserve reflections. There are two kinds of such automorphisms, namely 
s-transvections and J- local automorphisms. 

s-Transvections. Let (W, S) be a Coxeter system and let s 6 S. We define 
the odd connected component of s in the diagram T(S) to be the set of all elements 
f e S for which there is a path from s to t such that all its edge-labels are odd. We 
denote the odd component of s by odd(s) and we put 

eodd(s) := odd(s) U {t <E S | o(it') ^ 00 for some t' <E odd(s)}. 

Let J s denote the irreducible component of eodd(s) which contains s and let K s 
denote the union of all spherical irreducible components of eodd(s) which do not 
contain s. 

Let z be an element in the center of (K s ). We define the mapping 9 S)Z : S —> W 
by setting 9 s . z (t) = tz if t £ odd(s) and by setting 9 S;Z (t) = t for the remaining 
t 6 S. One readily verifies that this mapping extends to an involutory automor- 
phism of W and that sz is not contained in S . Hence 9 StZ {S) is a Coxeter 
generating set of W providing a different set of reflections. 

The involutory automorphism described above is called an s-transvection of the 
Coxeter system (W, S). In fact, the definition of an s-transvection given in |27| is 
slightly more general. This is due to particular instances which might arise when 
there are subsystems of type C3. Due to these instances the formal definition of 
an s-transvection is somewhat involved and will be omitted here. Nevertheless, we 
give an example of such a C3-transvection because - unlike for the other kinds of 
automorphisms - it is not an 'obvious automorphism easily seen from the diagram'. 
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Example Let (W, S) be a Coxeter system where S = {s, t, t', c] such that o(st) = 
o(st') = 3, o(ct) = o(ct') = 4, o(sc) = 2 and o(tt') = oc. Define 6> : S -> W by 
setting 0(c) := c, 0(s) := sc, 0(f) := stcsts and 0(t') := st'cst's. One verifies that 
extends uniquely to an involutory automorphism of W. 

J-local automorphisms. Let (W, S) be a Coxeter system. A subset J of S is 
called a graph factor of (W, S) if J is spherical and if for alH E S \ J either tj = jt 
for all j E J or o(tj) = oo for all j € J. 

Let J be a graph factor of (W, S 1 ) and let a be an automorphism of ( J). Then 
it is readily verified that there is a unique automorphism of W stabilizing the 
subgroup (J), inducing a on it and inducing the identity on S\J. We call such an 
automorphism a J-local automorphism. 

This observation can be used to produce non-reflection preserving automor- 
phisms. There are lots of examples of finite Coxeter groups, having automorphisms 
which are not reflection preserving. Obvious examples are the elementary abelian 
2-groups. A particularly interesting example is of course the exceptional automor- 
phism of Sym(6) which is the Coxeter group of type A§. 

The finite continuation of a reflection. Let (W, S) be a Coxeter system. 
As S is supposed to be finite and as each finite subgroup of W is conjugate to a 
subgroup of some spherical standard parabolic subgroup it follows that there is an 
upper bound for the order of any finite subgroup of W . This implies that there is 
for any subgroup X of W a unique maximal normal finite subgroup of X which we 
denote by Ofi n (A). 

Let r 6 W be an involution of W; by the result of Richardson mentioned in 
Section|21(Fact 4) we know that r is conjugate to some pj for some spherical subset 
J of S and such that p,j is central in (J). Now one knows that N\y((J)) = Cw{pj) 
(Fact 5) and hence (J) is contained in Ofi n (Cw(pj))- These considerations show 
that r must be a reflection if Ofi n (Cw(r')) = (r). Hence we have found a handy 
criterion which ensures that a given involution of an abstract Coxeter group is a 
reflection for any Coxeter generating set of that group. 

This idea was the starting point for the results obtained in |23j . It soon turned 
out that it is more convenient to work with the finite continuation FC(r) rather than 
with the group Ofi n (CV(r)). This is defined to be the intersection of all maximal 
finite subgroups of W containing r. The main result of |23) is the following theorem. 
Its proof is based on a careful analysis of the centralizer of a reflection which had 
been desrcibed in detail in 

Theorem 4.1 Let (W, S) be a Coxeter system and let s G S. Then FC(s) is 
known. Moreover, ifFC(s) = (s) , then s is a reflection for each Coxeter generating 
set of W. 

The description of FC(s) may become complicated if there are subsystems of 
type C3 or D4. If this is not the case, one can describe FC(s) by means of the 
subsets J s and K s defined in the paragraph on s-transvections as follows. 

Corollary 4.2 Let (W, S) be a Coxeter system and suppose that there is no 
subsystem of type C3 or D4. Let s £ S. If J s is spherical, then FC(s) = (J s U K s ); 
in the remaining cases one has FC(s) = {{s} U K s ). In particular, if K s = and 
J s is non- spherical, then s is a reflection for each Coxeter generating set of W . 
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The reduction theorem. Let (W, S) be a Coxeter system. We call s G S 
FG-centered if FC(s) = (J) for some J C S. A fundamental reflection might not 
be FC-centered if there are subsystems of type C3 or D4. Moreover, the group of 
automorphisms of W which stabilize the subset S w is denoted by Keis(W). We 
are now able to state the main result of |27) . 

Theorem 4.3 Let (W, S) be a reduced Coxeter system. For each FC-centered 
s G S, let T s denote the group of all s-transvections of (W,S). For each graph 
factor J G S let Lj denote the group of all J -local automorphisms of (W,S). Let 
E be the subgroup of Aut(W) which is generated by all T s and all Lj, where s runs 
through the FC-centered elements of S and J runs through the set of graph factors 
of (W,S). Let E be the subgroup of Aut(W) which stabilizes FC(s) for all s G S. 
Then we have the following: 

a) The group E is finite and E < E. In particular, E is a finite subgroup of 
Aut(W). 

b) Given a reduced Coxeter system (W , S') and an isomorphism a : W — > W , 
then there exists a G E such that a(o{S)) C S lW . 

c) The group E (and hence also the group EJ is a finite supplement o/Rcfs(PF) 
in Aut(W). 

Part b) of the theorem above says in particular, that if (W, S) and (W , S') are 
Coxeter systems which are both reduced and if there is an isomorphism from W 
onto W , then there is also an isomorphism between them which maps S w onto 
S lW . This yields the reduction of Problem 1 to Problem 3 for reduced Coxeter 
systems. Moreover, given any reduced Coxeter system (W, S), then its group of au- 
tomorphism can be written as ERefs(VF), hence Problem 2 is reduced to Problem 
4 for reduced Coxeter systems. 

5 The restricted isomorphism problem 

In view of the reduction result described in the previous section it suffices to 
solve Problems 3 and 4 in order to solve Problems 1 and 2 respectively. Thus we 
are led to the following question. 

Question: Let (W, S) be a Coxeter system and let R C S w be a Coxeter generating 
set ofW. What can be said about R? 

We have to consider Coxeter generating sets whose elements are reflections in 
a given Coxeter system. The following is a first observation which can be shown by 
using the geometric representation of a Coxeter group. 

Lemma 5.1 Let (W, S) be a Coxeter system, let R C S w be a Coxeter gener- 
ating set of W and let X C R be such that (X) is finite. Then there exists a subset 
J of S and an element w G W such that (X) w — (J). In particular, if 'r,r' G R are 
such that o(rr') = n =/= 00, then there exist s, s' <E S such that o(ss') = n and such 
that the subgroups (r, r 1 ) and (s,s') are conjugate. 

Let (W, S) be a Coxeter system and let R C S w be a Coxeter generating set. 
We call R sharp-angled with respect to S if for any two reflections r,r' G R there 
exists w G W such that {r, r'} w C S. 

Let W be the dihedral group of order 2n for some natural number n > 2. We 
consider W as the group of automorphisms of the regular n-gon in the euclidean 
plane. Let S — {s,t}, where s and t are reflections whose axes intersect in an 
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angle ^. Given r ^ r' G then {r, r'} is sharp-angled with respect to S if the 
reflection axes of r and r' intersect in an angle — . 

Angle-deformations. Let (W, S) be a Coxeter system, let s ^ t 6 S be such 
that st has finite order, let x 6 (s,t) be such that (s^xtx^ 1 ) = (s,t) and put 
y := S \ ({s, i} U {s, t} ). Let F s be the set of all y G Y for which there exists a 
sequence yi, . . . , yu = y in Y such that o(syi), o(yiy 2 ), ■ ■ ■ , o(y k -iyk) are finite and 
define Y t analogously. We define the mapping S x : S — > W by setting <5^(r) := r if 
reS\(y t U {<}) and S x (r) — xrx^ 1 in the remaining cases. The following is easy 
to verify. 

Lemma 5.2 If Y s HY t = 9 then 5 X extends uniquely to an automorphism of 
W which stabilizes the set S w . 

If {s, xtx~ x } is not sharp-angled with respect to {s, t} and if 8 X is as above, then 
8 X (S) is not sharp-angled with respect to S. We therefore call the automorphisms 
of the lemma above angle-deformations. 

The following result can be obtained by using rigidity of Fuchsian Coxeter 
groups in a similar way as it was done in |38| . 

Proposition 5.3 Let (W, S) be a Coxeter system and suppose that there is no 
3-subset J of S such that M(J) = H^. Let A be the group generated by all angle 
deformations of (W,S). Given a Coxeter generating set R C S , then there exists 
5 e A such that S(R) is sharp-angled with respect to S . 

In view of the previous proposition the following conjecture is known to be true 
for Coxeter systems having no subsystem of type H3. 

Conjecture 1: Let (W, S) be a Coxeter system and R C S w be a Coxeter gener- 
ating set. Then there exists an automorphism aofW such that ct(S w ) — S w and 
such that ct(R) is sharp-angled with respect to S. 

Twist-equivalence. Let (W, S) be a Coxeter system and let R C S w be a 
Coxeter generating set of W. Recall that R 1 C S w is called a twist of R if there 
is an i?-admissible pair {J,K) such that R' = Trj^-)(R). Moreover, R' is a twist 
of R if and only if R is a twist of R 1 . By taking the transitive closure we obtain 
an equivalence relation on the set of the Coxeter generating sets contained in S w 
which is called twist- equivalence. 

If R' is a twist of R C S w , then R 1 C R w and R 1 is sharp-angled with respect 
to R. Hence, if R 1 is twist-equivalent with R C S w , then R 1 C R w and R 1 is 
sharp-angled with respect to R. There is some evidence that the converse is also 
true. This is the content of the conjecture below. This conjecture is a refinement 
of Conjecture 8.1 in |1U| . 

Conjecture 2: Let (W, S) be a Coxeter system and R C S w a Coxeter generating 
set ofW which is sharp-angled with respect to S. Then R is twist-equivalent to S. 

At present, the following two theorems are known by recent work of P.-E. 
Caprace. The first improves earlier results obtained in jlO) . and 38 . 

Theorem 5.4 Conjecture 2 holds for all Coxeter systems which do not contain 
an irreducible spherical subsystem of rank 3. 

Theorem 5.5 If (W,S) is a Coxeter system such that M(J ) is 2-spherical 
for each spherical subset J of S, Conjecture 2 holds for (W, S). 
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The main tool to prove Conjecture 2 in the references above is known to the 
experts as 'Kac Conjugacy Theorem for root bases'. This theorem is proved in [311 
for afhne and compact hyperbolic groups. A proof for all Coxeter groups is given 
in [28] . 

6 The solution of Problem 1 

Let M be a Coxeter diagram over a set /. Recall that M' is called a twist of 
M if there is a twist I' of / C W{M) such that M(J') is isomorphic to M' . Again, 
M' is a twist of M if and only if M is a twist of M' and by taking the transitive 
closure we obtain an equivalence relation on the set of Coxeter matrices which is 
called twist-equivalence as well. 

The following lemma is easy to prove. 

Lemma 6.1 Let (W,S) be a Coxeter system and let M be a Coxeter matrix. 
Then the following are equivalent. 

a) There exists a Coxeter generating set R C S w such that M(R) is isomorphic 
to M and such that R is twist- equivalent to S . 

b) The matrices M(S) and M are twist- equivalent. 

Using the previous lemma one obtains the following theorem, which yields the 
solution of Problem 3. 

Theorem 6.2 Let (W, S) and (W, S') be Coxeter systems and suppose that 
Conjectures 1 and 2 hold for (W, S). Then the following are equivalent. 

a) M(S) and M(S') are twist- equivalent. 

b) There exists an isomorphism a : W' — > W such that a(S') C S w 

We recall that a Coxeter system (W, S) is reduced if the set S contains no 
pseudo-transposition, that there is a natural notion of a Coxeter system or a Coxeter 
matrix to be a reduction of another and that it is always possible to produce a 
reduced reduction of a Coxeter system or Coxeter matrix by an easy algorithm. 
Now the previous theorem and Theorem 14 . 31 yield the following. 

Theorem 6.3 Let M and M' be irreducible Coxeter matrices of rank at least 
3 and let (W, S) be a Coxeter system of type M. If Conjectures 1 and 2 hold for 
(W,S), then the following are equivalent. 

a) The groups W{M) and W(M ) are isomorphic. 

b) If Mi is a reduced reduction of M and if M[ is a reduced reduction of M' , 
then Mi and M[ are twist equivalent. 

In view of Theorem 15 .41 and Proposition 15.31 we have the following corollary. 

Corollary 6.4 Let M and M 1 be Coxeter matrices and suppose that M has 
no subdiagram of type A3, C3 or H%, then the following are equivalent: 

a) The groups W(M) and W(M') are isomorphic. 

b) If Mi is a reduced reduction of M and if M[ is a reduced reduction of M' , 
then Mi and M[ are twist equivalent. 

7 On automorphisms of Coxeter groups 

The previous section shows that there is — under the hypothesis that Conjec- 
tures 1 and 2 are true — a satisfactory solution of Problem 1. Unfortunately, we can- 
not offer a satisfactory description of the automorphism groups of Coxeter groups 
under the same assumptions which would yield a solution of Problem 2 as well. In 
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fact, the author has serious doubts whether such a handy description exists in the 
general case. Nevertheless there are several natural subgroups of the automorphism 
group of a Coxeter group which are quite well understood. In most of the 'interest- 
ing' cases, the understanding of these subgroups suffices to understand the group 
of automorphisms as a whole. Our discussion will be restricted to those subgroups. 
Before going more into the details we would like to mention that the automorphism 
groups of Coxeter groups had been determined in various special cases. 

1. A presentation of the automorphism groups of right-angled Coxeter groups 
was given in |36| . This work was based on the results obtained in |45| and 
the latter is a far reaching generalization of the result in |30j . 

2. The automorphism groups of 2-spherical Coxeter groups are 'trivial' (i.e. 
all automorphisms are inner-by-graph) if there is no direct factor which is 
spherical. This result was accomplished in |14| and |23| . A 'virtual' result in 
this direction has been obtained already in ;28 and the main tool developed 
there was used again in |14j . 

3. The automorphism groups of several classes of Coxeter groups which are 
'almost spherical' have been described in |19j . |2(Jj . |21| and [22 . In 22 a 
complete description of the automorphism groups of the irreducible spherical 
Coxeter groups is given. 

Given an abstract Coxeter group W, then there is always a Coxeter generating 
set S C W such that (W, S) is reduced. Thus, there is no loss of generality if we 
consider only reduced Coxeter systems in this section. Let (W, S) be a reduced 
Coxeter system. We define the following subgroups: 

1. Rafs(W) := {a G Aut(VK) | a(S w ) = S w }, 

2. Ang s (W) := {a E Refg(W) | a(S) sharp-angled with respect to S}, 

3. t s (W) := {a G Aut(W) | a(FC(s)) = FC(s) for all s G S} 

4. T S (W) := {a G Aut(W) | a(S) = S} 

In view of Theorem 14.31 we have Aut(W) = £s(VF) Ref s(W) and the group 
T,s(W) is a finite group. Thus, there is a finite supplement of Ref s(W) in Aut(W). 
There is the natural question about minimal supplements (or even complements) 
of Ref s(W) in Aut(W). The example of the Coxeter group of type A*l shows that 
there are not always complements. However, a careful analysis of several special 
cases provides some evidence for the following conjecture. 

Conjecture 3: Let (W, S) be a reduced Coxeter system. Then there exists a 
subgroup Q < t s (W) such that II := Q D Ref s (W) < T S (W) and such that ft is 
a supplement of Ref s (W) in Aut(W). Moreover, there is a normal 2-subgroup U 
of f2 and a complement of L of U in such that L — L\ x L 2 x • • • Lk where Li 
is isomorphic to GL(rij, 2) for some natural number ni for 1 < i < k and II PI Li is 
just the set of permutation matrices. 

There is a canonical candidate for the choice of the group fl and based on this 
choice the validity of the conjecture is not difficult to see in several special cases. 
However, the arguments become somewhat involved in the general case. 

Reflection-preserving automorphisms. As Ref s{W) has a finite supple- 
ment, a big part of Aut(W^) is understood if Ref s(W) is understood. A first ob- 
servation is that Ang s (W) is a normal subgroup of finite index in Ref s(W) and 
therefore a similar remark holds for Ang s (W). We do not know whether Ang s (W) 
always has a finite supplement in Ref s(W) but we believe that there are examples 
where this is not the case. If there is no iJ3-subdiagram, then the group Ref s(W) 
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is generated by the angle-deformations of (W, S) and Ang s (W). We expect this to 
be true in general with a suitable definition of angle-deformations in the case where 
there are £/3-subdiagrams. 

In the following we will consider the group Ang s (W). Let 

R := {R C S w | R sharp-angled Coxeter generating set of W with respect to S} 

and call two elements R ^ R' in R adjacent if one is a twist of the other. This 
yields a graph which we call C. Conjecture 2 is equivalent to the statement that 
the graph C is connected. 

We consider first the special case where M(S) is even in which case Conjectures 
1 and 2 are known to be true. If M(S) is even, there is for each neighbor R of 
S in the graph C a canonical involution Or in Ang s (W) which switches S and R. 
Setting X :— {Or \ R neighbor of S), one verifies that C is the Cayley graph of X 
with respect to this generator set and that Ts is a complement of X in Ang s (W). 
It is probably possible to generalize the arguments given in |36j in order to give a 
presentation of the group Ang s (W). The key ingredient of such a generalization 
would be the observation that the group Ang s (W) is something like a 'generalized 
Coxeter group' as it is in the right-angled case. 

Let's consider the general case under the assumption that Conjecture 2 holds. 
The situation becomes more complicated. The graph C is no longer the Cayley 
graph of a group but of a groupoid. We do not go into the details here. But 
it is worth mentioning that a similar situation occurs if one is interested in the 
normalize!' of a parabolic subgroup in a Coxeter group. These normalizers had 
been described in [8] and |12| in a satisfactory way. The key observation in |12| is 
that they are finite index subgroups of a groupoid which one might call a Coxeter 
groupoid in view of its properties which are quite similar to those of Coxeter groups. 
We believe, that a presentation of Ang s (W) can be given by using analogous ideas. 
It would be based on the observation that the graph C is the Cayley-graph of a gen- 
eralized Coxeter groupoid of which Ang s (W) is a subgroup of finite index. However, 
a concrete description of such a presentation might become rather involved. 
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